Consensus-based In-Network Computation of the PARAFAC Decomposition

Higher-order tensor analysis is a multi-disciplinary tool widely used in numerous application areas involving data analysis such as psychometrics, chemometrics, and signal processing, just to mention a few. The parallel factor (PARAFAC) decomposition, also known by the acronym CP (standing for “CANDECOMP/PARAFAC” or yet “canonical polyadic”) is the most popular tensor decomposition. Its widespread use comes from its essential uniqueness property under mild conditions as well as to the existence of several numerical algorithms that can be used to compute the decomposition. In this work, we present a new approach for the distributed computation of the PARAFAC decomposition of a thirdorder tensor across a network of collaborating nodes. We are interested in the case where the overall data gathered across the network can be modeled as a data tensor admitting an essentially unique PARAFAC decomposition, while each node only observes a sub-tensor with not necessarily enough diversity so that identifiability conditions are not locally fulfilled at each node. In this situation, conventional (centralized) tensor based methods cannot be applied individually at each node. By allowing collaboration between neighboring nodes of the network, we propose distributed versions of the alternating least squares (ALS) and Levenberg-Marquardt (LM) algorithms for the in-network estimation of the factor matrices of a thirdorder tensor. We assume that one of the factor matrices contains parameters that are local to each node, while the two remaining factor matrices contain global parameters that are common to the whole network. The proposed algorithms combine the estimation of the local factors with an in-network computation of the global factors of the PARAFAC decomposition using average consensus over graphs. They emulate their centralized counterparts in the case of ideal data exchange and ideal consensus computations. The performance of the proposed algorithms are evaluated in both ideal and imperfect cases.